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Open Court Reading® 

Program Description 1 

Open Court Reading © is a core reading program for grades K-6 
developed by SRA/McGraw-Hill that is designed to teach decoding, 
comprehension, inquiry, and writing in a logical progression. Part 1 
of each unit, Preparing to Read, focuses on phonemic awareness, 
sounds and letters, phonics, fluency, and word knowledge. Part 2, 
Reading and Responding, emphasizes reading literature for under- 
standing, comprehension, inquiry, and practical reading applications. 
Part 3, Language Arts, focuses on writing, spelling, grammar, usage, 
mechanics, and basic computer skills. SR A/McGraw-Hill revised 
Open Court Reading © and changed the name to Imagine It!© in 2007. 2 
The study featured in this report evaluates Open Court Reading ©. 

Research 3 

One study of Open Court Reading © that falls within the scope of the 
Adolescent Literacy review protocol meets What Works Clearing- 
house (WWC) evidence standards without reservations. The study 
included more than 900 first-grade through fifth-grade students 4 who 
attended five schools located in five states across the United States. 
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Based on this study, the WWC considers the extent of evidence for 

Open Court Reading © on adolescent readers to be small for one domain: comprehension. Three other domains are 
not reported in this intervention report. (See the Effectiveness Summary for further description of all domains.) 


Effectiveness 

Open Court Reading © was found to have potentially positive effects on comprehension for adolescent readers. 


Table 1. Summary of findings 5 




Improvement index (percentile points) 




Outcome domain 

Rating of effectiveness 

Average 

Range 

Number of 
studies 

Number of 
students 

Extent of 
evidence 

Comprehension 

Potentially positive effects 

+6 

na 

1 

917 

Small 


na = not applicable 
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Program Information 

Background 

Open Court Reading ® was developed for and is distributed by SRA/McGraw-Hill. SRA/McGraw-Hill revised the 
program and changed the name to Imagine It!® in 2007. Additions made to the program that came with the revi- 
sion include increased instruction in vocabulary, writing and inquiry, stronger support for English language learners, 
and enhanced technology components. Address: McGraw-Hill Education, P.O. Box 182605, Columbus, OH 43218. 
Web: https://www.mheonline.com/. Telephone: (800) 334-7344. 

Program details 

Open Court Reading® materials are divided by grade and include the Reading, Phonemic Awareness and Pho- 
nics Kit (K); Sounds and Letters Workbook (K); Language Arts Skills Workbook (K); Big Books and Little Books 
(K— 1 ); Language Arts Big Book (K— 1 ); Pre-Decodable and Decodable Texts (K-3); Part 1 Lesson Cards (K— 3); Desk 
Strips (K-3); Unit Assessment (K-6); Transparencies (K-6); Writer’s Workbook (K-6); Challenge Workbooks (K-6); 
Reteach Workbooks (K-6); Intervention Support (K-6); Phonics Skills Workbook (1); First and Second Readers 
(1-2); Reading and Phonics Package (1-3); Student Anthologies (1-6); Comprehension and Language Arts Work- 
book (1-6); Spelling and Vocabulary Skills Workbook (1-6); Inquiry Journal (2-6); and Language Arts Handbook 
(2-6). The Teacher’s Edition (K-6) contains information on providing systematic, explicit skills instruction centered 
on literature selections. Lesson plans indicate the goals and objectives for each lesson and detailed suggestions 
for carrying out the lessons. 


Cost 

The Open Court Reading® curriculum includes grade-specific student textbooks, workbooks, decodable books, 
and anthologies. Open Court Reading® Online Professional Development provides support for teachers, principals, 
reading specialists, and coaches. For details on specific product pricing, contact the program developer, 
SRA/McGraw-Hill. 
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Research Summary 


Fifty-eight studies reviewed by the WWC investigated the effects 
of Open Court Reading ® on adolescent readers. One study (Bor- 
man, Dowling, & Schneck, 2008) is a randomized controlled trial 
that meets WWC evidence standards without reservations. This 
study is summarized in this report. The remaining 57 studies do 
not meet either WWC eligibility screens or evidence standards. 
(See references beginning on p. 5 for citations for all 58 studies.) 

Summary of study meeting WWC evidence standards without 
reservations 


Table 2. Scope of reviewed research 


Grade 

1,2, 3, 4, 5 

Delivery method 

Whole class 

Program type 

Curriculum 

Studies reviewed 

58 

Meets WWC standards 
without reservations 

1 study 

Meets WWC standards 
with reservations 

0 studies 


Borman et al. (2008) conducted a randomized controlled trial that 

examined the effects of Open Court Reading ® on first-grade through fifth-grade students attending five schools 
from five states during the 2005-06 school year. At each school, classrooms were randomly assigned within each 
grade either to implement Open Court Reading ® or to serve as the comparison group. The WWC based its effec- 
tiveness rating on findings from 917 students from grades 1-5 who participated in the study; 507 in the Open Court 
Reading® group and 410 in the comparison group. The study reported student outcomes after approximately seven 
months of program implementation. 


Summary of studies meeting WWC evidence standards with reservations 

No studies of Open Court Reading ® meet WWC evidence standards with reservations. 
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Effectiveness Summary 

The WWC review of interventions for Adolescent Literacy addresses student outcomes in four domains: alpha- 
betics, reading fluency, comprehension, and general literacy achievement. The one study that contributes to the 
effectiveness rating in this report covers one domain: comprehension. The comprehension domain includes two 
constructs: reading comprehension and vocabulary development. The findings below present the authors’ esti- 
mates and WWC-calculated estimates of the size and statistical significance of the effects of Open Court Reading ® 
on adolescent readers. 6 For a more detailed description of the rating and extent of evidence criteria, see the WWC 
Rating Criteria on p. 16. 

Summary of effectiveness for the comprehension domain 

One study reported findings in the comprehension domain. 

Borman et al. (2008) found, and the WWC confirmed, statistically significant effects of Open Court Reading ® 
on the Reading Composite score of the Comprehensive Test of Basic Skills, 5th edition (CTBS/5) Terra Nova test. 

Thus, for the comprehension domain, one study showed statistically significant positive effects of Open Court 
Reading®. This results in a rating of potentially positive effects, with a small extent of evidence. 


Table 3. Rating of effectiveness and extent of evidence for the comprehension domain 


Rating of effectiveness 

Criteria met 

Potentially positive effects 

Evidence of a positive effect with 
no overriding contrary evidence. 

The review of Open Court Reading in the comprehension domain had one study showing statistically significant 
positive effects and no studies showing statistically significant or substantively important negative effects or 
indeterminate effects. 

Extent of evidence 

Criteria met 

Small 

The review of Open Court Reading in the comprehension domain was based on one study that included five 
schools and 917 students. 
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Appendix A: Research details for Borman et al., 2008 

Borman, G. D., Dowling, N. M., & Schneck, C. (2008). A multi-site cluster randomized field trial of Open 
Court Reading. Educational Evaluation and Policy Analysis, 30(4), 389-407. 

Table A. Summary of findings Meets WWC evidence standards without reservations 


Study findings 
Average improvement index 

Outcome domain Sample size (percentile points) Statistically significant 


Outcome domain 

Study findings 
Average improvement index 

Sample size (percentile points) Statistically significant 

Comprehension 

49 classrooms/917 students +6 Yes 

Setting 

The study initially included six schools— one each in Florida, Georgia, Idaho, Indiana, North 
Carolina, and Texas. Two schools were from rural areas, two from suburban areas, and two 
from urban areas. The Georgia school dropped out of the study. 

Study sample 

SRA/McG raw- Hill recruited a group of schools that had not previously used Open Court 
Reading ® to participate in the study. The six schools that initially participated were given free 
Open Court Reading © materials as well as a training program for teachers and implementation 
support. At each school, classrooms were randomly assigned within each grade either to be 
enrolled in Open Court Reading © or to serve as the comparison group. 

The initial sample consisted of 57 grade 1-5 classrooms containing a total of 1 ,099 students. 
The Georgia school dropped out of the study, which resulted in a loss of four classrooms in both 
the intervention and comparison groups. Some students were absent during the administration 
of the posttest. However, the resulting attrition rates of schools and classrooms were low. 7 

The analysis sample consisted of students in grades 1-5 and included 507 students in the 27 
Open Court Reading © classrooms and 410 students in the 22 comparison classrooms. Par- 
ticipating students were more than 70% minority, and more than 75% were eligible for free or 
reduced price lunches. Fewer than 15% were English as a Second Language (ESL) students, 
and fewer than 10% were special education students. 

Intervention 

group 

Open Court Reading © is a curriculum that includes textbooks, workbooks, decodable books, 
and anthologies. The curriculum consists of three main components: (a) Preparing to Read, 

(b) Reading and Responding, and (c) Language Arts. Teachers were given a teacher’s edition 
of the curriculum that included scripted direct instruction lessons; and diagnostic and assess- 
ment packages. The program is designed to be used for 2.5 hours per day with grades 1-2 
and for two hours per day with grades 4-6. However, the authors report that external consul- 
tants observed that some teachers provided only 90 minutes of daily instruction. The interven- 
tion was implemented from fall to spring during the 2005-06 school year. 

Comparison 

group 

The comparison classrooms used a “business as usual” curriculum and were instructed not to 
use Open Court Reading © or any of its materials. Principals mentioned that curricula currently 
in use by the comparison classrooms consisted of Reading Street by Scott Foresman, Literacy 
Place by Scholastic, McGraw-Hill Reading by MacMillan/McGraw-Hill, Collections by Har- 
court, and Trophies by Harcourt. Consultants visited comparison classrooms and verified that 
they were not using Open Court Reading ©. 
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Outcomes and 
measurement 


Support for 
implementation 


For both the pretest (October 2005) and the posttest (May 2006), students took the Compre- 
hensive Test of Basic Skills, 5th edition (CTBS/5) Terra Nova Reading Comprehension and 
Vocabulary subtests. A Reading Composite score also is reported, which is the average of 
these two subtest measures. For a more detailed description of these outcome measures, 
see Appendix B. Findings for the combined student sample on the Reading Composite score 
can be found in Appendix C. Additional findings reflecting subtest outcomes for the combined 
student sample and separately for grades 4 and 5 can be found in Appendix D. 

Teachers were provided training opportunities with external consultants, which consisted of 
two- to three-day summer workshops. In addition, the consultants, who had teaching experi- 
ence and detailed knowledge of Open Court Reading ® and were trained by SRA/McGraw-Flill, 
visited and observed classrooms and provided feedback to teachers. 
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Appendix B: Outcome measures for each domain 


Comprehension 

Comprehensive Test of Basic Skills 
(CTBS/5) Terra Nova Reading Composite 
score 

This assessment consists of two subtests, Reading Comprehension and Vocabulary, and combines selected- 
response items with constructed-response items that allow students to produce short and extended responses. 
The Reading Composite score is a simple average of the CTBS/5 Reading Comprehension and Vocabulary 
subtests described below (as cited in Borman et al., 2008). 

Reading comprehension construct 

Comprehensive Test of Basic Skills 
(CTBS/5) Terra Nova Reading 
Comprehension subtest 

This assessment combines selected-response items with constructed-response items that allow students to 
produce short and extended responses. The Reading Comprehension subtest items focus on five objectives: 
(a) oral comprehension of passages read aloud, (b) basic understanding of literal meanings of passages, (c) 
analyzing text, (d) evaluating and extending meaning, and (e) identifying reading strategies (as cited in Borman 
et al, 2008). 

Vocabulary development construct 

Comprehensive Test of Basic Skills 
(CTBS/5) Terra Nova Vocabulary subtest 

This assessment combines selected-response items with constructed-response items that allow students to 
produce short and extended responses. The Vocabulary subtest focuses on three objectives: (a) understand- 
ing word meaning, (b) identifying multi-meaning words, and (c) inferring words in context (as cited in Borman 
et al, 2008). 
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Appendix C: Findings included in the rating for the comprehension domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Borman et al., 2008 a 

CTBS/5: Reading Composite 
score 

Grades 

1-5 

49 classrooms/ 
917 students 

612.77 

(49.36) 

604.82 

(28.55) 

7.95 

0.16 

+6 

<0.05 


Domain average for comprehension (Borman et al., 2008) 0.16 +6 Statistically 

significant 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the change (measured in standard deviations) 
in an average student’s outcome that can be expected if the student is given the intervention. The improvement index is an alternate presentation of the effect size, reflecting 
the change in an average student’s percentile rank that can be expected if the student is given the intervention. The statistical significance of the study's domain average was 
determined by the WWC. The statistical significance of the domain average is the statistical significance of the hierarchical linear modeling (HLM) coefficient reported by the study 
authors for the Reading Composite score of the Comprehensive Test of Basic Skills, 5th edition (CTBS/5). 

a For Borman et al. (2008), no corrections for clustering or multiple comparisons were needed. The p-value presented here was reported in the original study. The effect size in the 
table is based on the HLM-adjusted results for combined grades 1-5 reported in Table 4 of Borman et al. (2008) (rather than the unadjusted means and standard deviations reported 
in Table 3 of Borman et al. [2008], since this effect size does not control for pre-intervention test scores, and the p-value does not account for the clustered sample design). The 
intervention mean equals the comparison mean plus the intervention coefficient from the HLM analysis. 
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Appendix D: Supplemental subtest findings for the comprehension domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Borman et al., 2008 a 

CTBS/5: Vocabulary subtest 

Grades 

1-5 

49 classrooms/ 
918 students 

604.52 

(55.85) 

593.73 

(55.49) 

10.79 

0.19 

+8 

<0.05 

CTBS/5: Comprehension 
subtest 

Grades 

1-5 

49 classrooms/ 
923 students 

621.04 

(48.11) 

615.18 

(47.92) 

5.86 

0.12 

+5 

<0.05 

CTBS/5: Vocabulary subtest 

Grade 4 

7 classrooms/ 
133 students 

631.08 

(41.98) 

631.34 

(50.65) 

-0.26 

-0.01 

0 

>0.05 

CTBS/5: Comprehension 
subtest 

Grade 4 

7 classrooms/ 
138 students 

641.42 

(41.79) 

647.96 

(42.76) 

-6.54 

-0.15 

-6 

>0.05 

CTBS/5: Vocabulary subtest 

Grade 5 

6 classrooms/ 
105 students 

660.18 

(40.25) 

636.17 

(28.46) 

24.01 

0.69 

+25 

>0.05 

CTBS/5: Comprehension 
subtest 

Grade 5 

6 classrooms/ 
106 students 

662.53 

(33.04) 

650.84 

(38.18) 

11.69 

0.32 

+13 

>0.05 


Table Notes: The supplemental findings presented in this table are additional subtest findings from the study in this report that do not factor into the determination of the interven- 
tion rating. For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the change (measured in standard deviations) 
in an average student’s outcome that can be expected if the student is given the intervention. The improvement index is an alternate presentation of the effect size, reflecting the 
change in an average student's percentile rank that can be expected if the student is given the intervention. CTBS/5 = Comprehensive Test of Basic Skills, 5th edition. 

a For the Borman et al. (2008) combined sample results (grades 1-5), a correction for multiple comparisons was needed, but the WWC could not apply this correction because exact 
p-values were not reported by the authors. An author query was not performed to request exact p-values for the subtest results because the significance level of the findings in this 
study is based on the Reading Composite score (reported in Appendix C) for which no correction for multiple comparisons was needed. The first two effect sizes in this table are based 
on the HLM-adjusted results for combined grades 1-5 reported in Table 4 of Borman et al. (2008). The intervention mean equals the comparison mean plus the intervention coefficient 
from the HLM analysis. The p-value levels presented for the first two outcomes were reported in the original study. 

For the individual grade results (grade 4, grade 5), corrections for clustering and multiple comparisons were needed. When adjusted for clustering of students within classrooms, the 
WWC-calculated effects on the outcome measures listed in the table were not statistically significant. For the individual grade results, Borman et al. (2008) reported the unadjusted 
posttest means and standard deviations. The p-values were not reported in the original study. An author query was not performed to request these p-values because they would not 
have affected the significance level of the main results used in the WWC rating of the study. The reported p-values were computed by the WWC. 
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Endnotes 

1 The descriptive information for this program was obtained from a publicly available source: the program’s website (https://www. 
sraonline.com/productsamples. html?show=2&gid=342&tid=1 , downloaded June 201 1). The WWC requests developers review the 
program description sections for accuracy from their perspective. The program description was provided to the developer in June 
201 1 , and we incorporated feedback from the developer. Further verification of the accuracy of the descriptive information for this 
program is beyond the scope of this review. The literature search reflects documents publicly available by August 201 1 . 

2 This information was obtained from a publicly available source: http://www.mcgraw-hill.com/releases/education/20070502.shtml 
(downloaded August 2011). Additions made to the program that came with the 2007 revision and renaming to Imagine It!® include 
increased instruction in vocabulary, writing and inquiry, stronger support for English language learners, and enhanced technology 
components. 

3 The studies in this report were reviewed using WWC Evidence Standards, version 2.1 , as described in the Adolescent Literacy review 
protocol, version 2.0. The evidence presented in this report is based on available research. Findings and conclusions may change as 
new research becomes available. 

4 The Adolescent Literacy topic area reviews studies of interventions administered to students in grades 4-12 (or 9-18 years of age). 
For studies that include samples of students that span both the Adolescent Literacy (grades 4-12) and Beginning Reading (grades 
K-3) topic areas and cannot be adequately disaggregated by grade level, the Adolescent Literacy topic area will review any studies 
that include fifth-grade students or higher (for example, a combined sample of students from grades 3-6). The Beginning Reading 
topic area will review any studies that include only fourth-grade students and lower (for example, a combined sample of students 
from grades K-4). Findings in this Adolescent Literacy report are based on results from the FILM analysis for a combined sample of 
students from grades 1-5 (Borman et al., 2008) because the HLM coefficients that were used to calculate effect sizes on the outcome 
measures in Appendix C were not disaggregated by grade level in the original study. 

5 For criteria used in the determination of the rating of effectiveness and extent of evidence, see the WWC Rating Criteria on p.16. 
These improvement index numbers show the average and range of student-level improvement indices for all findings across the study. 

6 The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clus- 
tering within classrooms or schools and for multiple comparisons. For the formulas the WWC used to calculate the statistical signifi- 
cance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, 
Appendix D for multiple comparisons. In the case of Borman et al. (2008), a correction for multiple comparisons was needed for the 
two subtests included in Appendix C, but the WWC could not apply this correction because exact p-values were not reported by the 
authors. The statistical significance of the domain average is the statistical significance of the HLM coefficient reported by the study 
authors for the Reading Composite score of the Comprehensive Test of Basic Skills, 5th edition. 

7 Student attrition rates are unknown, as study authors did not attempt to match individual students from pretest to posttest. 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2012, August). 
Adolescent Literacy intervention report: Open Court Reading®. Retrieved from http://whatworks.ed.gov. 
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WWC Rating Criteria 

Criteria used to determine the rating of a study 


Study rating 

Criteria 

Meets WWC evidence standards 
without reservations 

A study that provides strong evidence for an intervention’s effectiveness, such as a well-implemented RCT. 

Meets WWC evidence standards 
with reservations 

A study that provides weaker evidence for an intervention's effectiveness, such as a QED or an RCT with high 
attrition that has established equivalence of the analytic samples. 

Criteria used to determine the rating of effectiveness for an intervention 

Rating of effectiveness 

Criteria 

Positive effects 

Two or more studies show statistically significant positive effects, at least one of which met WWC evidence 
standards for a strong design, AND 

No studies show statistically significant or substantively important negative effects. 

Potentially positive effects 

At least one study shows a statistically significant or substantively important positive effect, AND 

No studies show a statistically significant or substantively important negative effect AND fewer or the same number 

of studies show indeterminate effects than show statistically significant or substantively important positive effects. 

Mixed effects 

At least one study shows a statistically significant or substantively important positive effect AND at least one study 
shows a statistically significant or substantively important negative effect, but no more such studies than the number 
showing a statistically significant or substantively important positive effect, OR 

At least one study shows a statistically significant or substantively important effect AND more studies show an 
indeterminate effect than show a statistically significant or substantively important effect. 

Potentially negative effects 

One study shows a statistically significant or substantively important negative effect and no studies show 
a statistically significant or substantively important positive effect, OR 

Two or more studies show statistically significant or substantively important negative effects, at least one study 
shows a statistically significant or substantively important positive effect, and more studies show statistically 
significant or substantively important negative effects than show statistically significant or substantively important 
positive effects. 

Negative effects 

Two or more studies show statistically significant negative effects, at least one of which met WWC evidence 
standards for a strong design, AND 

No studies show statistically significant or substantively important positive effects. 

No discernible effects 

None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Criteria used to determine the extent of evidence for an intervention 

Extent of evidence 

Criteria 

Medium to large 

The domain includes more than one study, AND 
The domain includes more than one school, AND 

The domain findings are based on a total sample size of at least 350 students, OR, assuming 25 students in a class, 
a total of at least 14 classrooms across studies. 

Small 

The domain includes only one study, OR 
The domain includes only one school, OR 

The domain findings are based on a total sample size of fewer than 350 students, AND, assuming 25 students 
in a class, a total of fewer than 14 classrooms across studies. 
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Glossary of Terms 

Attrition 

Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Extent of evidence 

Improvement index 

Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Rating of effectiveness 

Single-case design 
Standard deviation 


Statistical significance 


Substantively important 


Attrition occurs when an outcome variable is not available for all participants initially assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review and inclusion in this report if it falls within the scope of the 
review protocol and uses either an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

An indication of how much evidence supports the findings. The criteria for the extent 
of evidence levels are given in the WWC Rating Criteria on p. 16. 

Along a percentile distribution of students, the improvement index represents the gain 
or loss of the average student due to the intervention. As the average student starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which subjects are assigned 
to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which investigators randomly assign 
eligible participants into intervention and comparison groups. 

The WWC rates the effects of an intervention in each domain based on the quality of the 
research design and the magnitude, statistical significance, and consistency in findings. The 
criteria for the ratings of effectiveness are given in the WWC Rating Criteria on p. 16. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample tend to be spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < 0.05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 2.1) for additional details. 
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